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ABSTRiCT • , \ ^ - 

Seven .guideline^ for setting iiniioi coipetency 

' standards for high 'School graduation are presented: (1) Coipetency 
staaiatds for the diploaa should be developed b% a\ process that gives 
»ajor attention to the "Heeds of students yet recognizes an i . 
institutions^s respo^nsibility to societal needs. {2\ toipetency 
standards should be /developed using good nan^geoient ^procedures. (3) 
Standards sust be set at realistic levelsl (4) Standards should be 
adflfinistered vith an adequate student *f]visory and guld^npe system 

/providing preparation and .feedback. * (5) Standards should be applied 
; eo&sistently vithin prograis> across prograss, and over tiie< (6) ' , 
/ standard? should be backed fully byjths hoie institution to increase* 
the lakelihpod that the resulting dipl^ia vill be acceptable to other 
institutions. (7) Standards should be administered so that the 
studen;bs involved feel they have been veil served. Three factors 
which are believed to influence a school district's actions afe the 
external requirenents of other Institutions and agencieSf the basic 
educatioival objective of the school district, and the personal 

. developaent goals pf individual students. Seven possible assessment 
protredur^s are outi|ned. They are: development of competence scales, * 

* use of the- traditional methods, r^viev of test questions, 

^ identification of pre^^quiisite skills, use of normative data, 
^ comparison of 'test scores to course performance, and use of 

* preselected groups. .(Author/HT) » ^ 
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BACKGROUND 

setting competepcy standards fox high scliool graduates 
has feaptured the attention of a wide variety of groups throughout the 
nation. The- most noticeable of these, of bourse, is the pres^. The 
^ Feb^iiary 28, 1977 issue of Time magazine, under the heading "Devalued 
\Dipl<m" indicates that: "... In the past year,^* 'mihlmal cdllipetency 
test'ing' has become the hottes^new catch phrase in public education."' 
As someone who has tried to keep up with developments in the area 
of minimal competency testing, my files are bulging with "news'' of this 
so*t.. the following hemlines tiaken from one newspaper. The Nev York • 
Times , provide ^ome Indication of the wiy the press is treating this ^ 
development: 

. • "Basic-Skill Tests Pushed for Blgh School Diploma^" March 14, 1976 

• "Minimum-Competency Tests for^tudentsr An Old Idea, Now Revive4 
and Spreading," January 31, W77 

• "Requiring Test to Get a D^loiia Brings Wide Attention to Dtove/," 
,n January 31, 1977 

• • "Certificates of High Scfcool Proficiency Stir Controversy," March 
' . 2, 1977 r 



r 




Ipaper presented at the annual meeting of tj^e National Council oi\y Meiaurement 
In Education, New Yotk City, April. 1977. (Part of Symposium— Credits, Theoriei 
and Standards: New Apilications* fot t^e Secondary Diploma] ^ 
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Educational journals and related publications Have also given th| 

% 

topic a great deal of attention. Some titles of articles and oopnentaries 
help illustrate some of the Issues that have been considered: 

' "The Politics of. Baek=^t;o-B^ics," Joan Baum, Change , November 
1976 (pp. 32-36).^ 

"Changing High School Requirements i " National Education Asso* 
elation, Brief ln| , Juhe 1976, No. 11. 

Competency Tests and Graduation Requirements , James 'P. Clark and 
Scott D. Thomson, National Association >of Secondary School 
Principals, 1976. 

So much has been' published on the topic. of mlnlmxm-competericy testing 
as a diploma requirement that I will not attempt to review all that is 
avail^le. Three sources however , warrant mentioning. The first is the 
National Association o£ Secondary School Principals (NASSP) . This organi- 
zation has given the most comprehensive attention that I have seen to the 
issue .of competency requlrem^ts and Uas gone on record .as favoring com- 
petency testing as one component of high school graduation requirements* ^ 
^ The second source is the Legislative Review , a pub 3^ at ion of the 
Education Commission of the States. The editor, Ch|?is Pipho, has done a 
conscientious job of collecting information about €ftate legislation and 
regulations regarding minimal competency testing. Mr. Pipho drew on this 
experience to give the following ^ords of advice to educators and legis- 
lators at the National Conference on Minimum Competencies in New York on 
March 4^ X977: * . 

To educatofs~Don* t underestimate the concern of the man In 

. « the street or of state legislators regarding 
f the skills .of high sdiool graduates. The* com- 

. petency requirement movement isi not the work 

I ol a few conservative legislators. 



To legislators — Don^t underestimate the technical problems 
, associated with legislation that calls for 

competency r^;^ulrement3 testing. 

^ The (hlrd source I will mexxtion is a document that I am using , as a' 
handout for this ^feting. It is aa Educational Testing Service Informa- 
tion Report entitled Basic Skills Assessment Arouad the Natlop . It pro- 
vides an overview of many^actlvltles at the^ state and district level 
relate^ to basic skills testing as a diploma requirement or, fn some cases 

4 

a grade- promotion requirement. ^ 

ISSUES RAISED BY THE COMPETENCY REQUIREMENTS MOVEMENT 
Other observers of the siinimum competeihcy movement have raised a 
number of issues. Among them are these: 

Meaning of the Diploma ' . ; 

There seems to 'be widespread Agreement tkat a high school diploma 

should be a guarantee of at least some minimal level of skill development 

on the part of the high school graduate. Reference is often made tcith^ 

^act that you can only get a "GED" (High Schl^l Equivalency Diploma) tiy 

passii^ the^ Tests of Geiietal Educatj.onal Development. Similar comments^ 

are being made about the California High School Proficiency Examination. 

* * 

Implications for Curriculign 

I' . ' ' 

Representatives professional associations, including the NCTE, 

NCTM, IRA, and NEA, have voiced concern that minimal comptency testing 

/ 

Vlll tend to* restrict the content of the high school program. A good^ 
statement of this position is contained in the December 1976 document 
'^Minimal Competencies and Measures ,of Competence" prepared by Alan Purves 



of the Uhiversity of Illinois. 

Impllcatloa for ^Student3 ' ) 

What ^out the stixdent who falls to meet competency standards fdr 

the high schdol diploma? There Is clear agreement that assessments should 

occur early in the secfbndary school years and that -remedial prqgrams should 

be available to students who fail to meet standards, ^ut will this result 

in tracking of students Into such programs and out' of the mainstream?) What 

about stydents .who never meet standards? What happen^ to them? • Does the 

school^ s responsibility end vhen stud^ts have completed four y^ars beyond 

grade eight? Wh^n the students r^ach a specified age?^ 
* 

Impllcatiotis for Measurement grofessionals ^ 

Should schools di^velop their own tests or use existing one^? What 
aireas should be as^ssed and with what types of tests? Can machine- ^ 
s-coreabl^ t^sts al<^e do the job? Hov should test-based and other infonOia- 
tion be combined to make decisions ^out individual students? (/ 

The qtestion of settin^and evaluating ^standards, therefore, has to be 
viewed as only one part ofN^ much larger set of iasties related to t^he intro- 
ductioh of 9ompetency standards. It is an. Important issue, though, with 
complexitlee tftat seemingly have not been recognized by many who have urged ^ 
the "adoption of jcompetency requirements for the diploma. 

GENERAL PROPERTIES Ot GOOD SYSTEMS FOR SETTING STANltos 
This section of my paper looks at the overall system of setting^ 
applying, and studying the effects of standards. Here is an overview of 

V % 

seven gulde}.lnes or principles that are reviewed: 
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r. Ethlcs/Responslblllty * - Canpetency standards for the diplqpia 
should be developed by a^ process that gives major attention 
to the needs of students but which* recognizes an institutions 's 
responsibility to "societal needs 

2. Mainakeaent Standards * Coinpetency standa^rds ^hould be. developed * 
i' using good management standai^ds or procedures. 



3. Reallsg . Standards oust be ,set at realistic levels. ' , 

4. Acieq^cy of Preparation ^and Feedback , Standards should be 

' » admals tared with ^ adequate student advisory and guidance 

eSjrst^ so that high standards are not accompanied by high 
♦ failure rates. . . 

5. Consistency . Standards should be applied consistently Within 
programs, across programs; and over time. ' 

6. Accep tab ility/TYansf erab llity . Standards should be backed fully 
, by the home institution to Increas^' the likelihood that the re~ 

suiting diploma will be acceptable to other institutions. 

7. Student Satisfaction . Standards should be administered so lhat 
the 'Students Involved feel they have been well served. 

Ethics/Res1>St>8ibllity . 

The issue of ethical responsibility is frequently Ignored when 
^educational systems and policies are evaluated. One simply asstimes that this 
issue is part of any such evaluation. However /this factor should ^be explic- 
Itly identified as a significant one. A commitment Xo meet' the educational 
needs of students should characterize all aspects of the standard-setting - 
process. Adherence toM:he guldjel^es 8uniaari;ced In this paper would consti7^ 
tute one type of evidence that a school or program, was meeting its responsi- 
bilities. Each institution, though, must analyzes its own situation to deter- 
mine the extent to which such constraints as limited time and money are dealt 
w:^th in ways that adequat/fely serve students. ^ ^ ' \ 

A commitmezit to meeting the educational needs of ^students does nof, ofx^ 
course, elimlnatS the jieed for schools to' consider the implications* for 



society of decisions made about individual ^students. As students satisfy 
degree requirements they moy^e toward the receipt of educational* tredent:(^ls . 
Keeton (1974, p. 3) makes the following observation about such credeittlals: 
"Creidelitlals are power Those who set the rules of the Qt^dentlaling game 
are establishing the* routes of access to that^ower/* ^ 

As institutions set the rules of the game, they must balance -the needs 

- ' . . ; ^ ' ^ \ ■ 

'of students and thQse of society. Whenever the completion^ of a particular 
program will carry the expectation that a student ha3 acquired certain 



competencies, an institution's evalilation procedures an^standards must be y 
adequate to certify t^e existence of such competencies. 

Management Standards ^ . . 

^ Turning first to the process by which criterion standards are initially 
established, it will be useful to ask whether an institution 4^s followed 
reasonable management standards. Management standards apply the steps 
caken in the design and implementation of . competency standards ,for the 
diploma, so that "necessary 'decisions are made by qualified people in a 
manner 'that provides f orvcons istent qjiality. In individual high school 
courses where perf orfilanceYt^ndard^' are set primarily by the teacher, ^ 
management standards apply Vo the selection of appropriate faculty menlbers. 
Additional management standards , relate to the' evaluation of that faculty ^ 
member's performance by his or. her princif^al, department chairman, and perhaps 
by the students. Since the Responsibility for establishing and applying com^ 
petency standards for the dipl6^ will not typically^be pldjc^d on a single 
•individual, ;tt will be necessary \to develop:^rocedures that tkke into account 
the nature and variety of contributing Individuals'. 

An Importaint first step in evalniating the adequacy of an institution's' 
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approach to stan^ard-sfettlng procedure i8\to determine vhether overall 

' . . ■ ' I 

responsibility for th^s task has been dtefined. The particular iadivldt;al(s) 

who hold this respons^illty can be ei^ected to vary depending on the nature 

of the program, -department, division, or other part of the Institution ' 

within which evaluations take place. Altjiough' someone with training and 

experience in the field of evaluation would be best for the job, those 

Without such background ar^ often selects. In such Instances, the use of 

trained evaluation consultants will be essential. , ' 

4 

Even when overall responsibility for establishing performance standards' 
can be placed la the hands ^ of a skilled evaluator, many other '^individuals 

should contribute to the standard- set ting process. For this purpose^ it 

/ 

will be useful to s^k assistance not only from faculty and students but also 

> > 
from membeirs of the larger community. \ 

m 

The job of designing and developing competency standard? cannot be 
handled adequately if decisions aqout particular students must'be made under 
great t:^e pressure. Providing adequate calendar time, and also adequate 
staff for .the development of competency standards, will make it possible to 
plan and carry out pilot projects to try out standards. setting systems. 
Procedures can be established J^ot monitoring the application of standards 
and foxf following' up on the students afterwards. Most Institutions will 

- - / o 

find It useful to adopt an iterative process fcjr developing competency stan- 
dards. The beat available sources of information should be used when setting 
standardis for the first group of students tested; The ejperience of con- 
ducting the initial evaluations, though, should become part of the background 
for future evaluations. 

John Valley of Educational Testing Service has sitggeatied the use of an 



outside "audit" as. part of an evaluation ol the adequacy of an Institution's 
management standards, fpr^ setting competency standards. The notion Is 
borrowed from the area of financial management. A well-manageti business ^ 
concern checks periodically the adequacy pt i.ts accountings-bookkeeping . 
procedures by calling for an outside audit.. The audit process consists of 
trac^g financial transactions from a variety of starting pplnts using 
different routes, different systems, and different people. The goal Is 
that of determlnin^vhether the same results can be obtained, as were pro- 
duMd by the company 's basic system. The application of ^he audit model 
to the assessment of student Aearnii^g against staxid^rd^ for the diploma 
would require the periodic use of ap ind^endent assessment method, fol- 
lowed«by»a comparison of ^assessment outcomes. '\ 

The foregoing guidelines for good standard-setting procedures relate ^ 
primarily to the' planning and managerial skills of the participating staff. 
There are other criteria that can be applied to the system they desJ 
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Critics of U.^. high schools often criticize the lack of rigor 
associated with the evaluation of dtudents' learning experleiihes . Little 
attention has been paid to the reverse problem^ — setting imreallstically - 
high standards. This pi^oblem is most likely to occur when the individual^* 
participating in the letting or a^llcation of standards do not have suf- 
ficlent direct knowledge of the performance of ^typical students^ It is the 
responsibility of the institution to judge what is reasWiable to expect . 
of stx^ents before making decisions regarding competency standards. 

school faculty and consultants from other Institutions or from 

V ~ . * * * 

the large^ commimlty should work together ^n' the planning and developing 



of assessmen^Attrocagiures. All parties concerned will need to collect 

\\\\ 

■^■^ V * ^ 

ana review ev^ae^f:e' of what the average student can reasonably e^^t to 
learn froip eac^i )||jf''^^the programs offered. Interviews with students' and 
reviews of atudipnw)^ers or products will be helpful ways of getting; an 
Idea of what I3 reaM&tlc to expect students. , . ' 

' Adequacy of Preparatlomtod Feedback ' t 

* ^ ^ M ' ' ^ 

Th^ student who ^ suflclently prepared for an evaluatloti ^gainst 

diploma standards should bAabi^to anticipate the result. The extent to 

V 

« which results will be predlAi^^ble will depend on the quality and ayall- 
abUlty of Information about \s'^^dards and on the effectiveness of^he 
student-advising process. Th^ a^^ssment syst^ and the standards employed 
can be considered unfair to stweSBs if they lead' to high ra^tes of failiire 
for students.who have worke^ coiist:^ntlously over^ a period of time. Since 
the focus of competency standards shdmld be on learning put comes, not on 



time spent or activities engaged In, this criterion may seem Inappropriate 
or parJfeoxlcal. Why should it pe considered unfair if a student has failed 
to meet a particular competency atandarm aftet working hard? What is ui^air 
is that the institution l^s not detected chat the student. was not^learning 
enou^ and has not acted accordingly. 

As Instlttttiohs gradually devel^^the^ assessment and standard-setting 
policies^ tjiey must pay attention to the coi^quences of a student's failure 

to meet a specified p^formance standard. Tttese consequences can range from 

• * • V ' » 1 

minor Inconveniences and disappoin^pent to major setbacks and disrt^ption^.' 

For example, if a student fails to meet a partlctxlar diploma standard when 

testejd in the ninth grade, his or tier failure n^y be of little moinent; The 

implications of failure, however, can be quite Cerent whin tha student Is' 
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already In the last year of .high school. > ' 

4!hQ reverse side of tiie issue of fairness, of course » Is that of 
qtiallty control. If award^ of ^he diploma Is not to be an automatic conse- 
quence of completing coui^e. and* attendance requirements, what steps can 
be taken both to maintain quality standards and still make failure a rare 
occurrence? Some form«of ongoing student evaluation^ and feedback "vlil best 

* 0 s « 

serve this need. Testing ne^ds to take place early enough so' that remedial 
rams pan b^ planned and carried out. Students can then sit»fot f6llow*up. 



examinations after they have further developed tii^ir skll]^ anci cpmpetencfls. 



Consistency • v 

la- Lnpl c m eriting established competency standards, 3chool districts will* 
have to achieve a high degree of consistency within and among schAl programs 
and across time* What can an institution do to promote consistency in the 
application of standards? One strategy is to conduct periodic reviews of 
the results of applying a particular •set of assessment procedure^ and s tan* 
dards. "If institutional guidelines for applying s%^dards «re not being 
followed, a' review of background materlals^ and guidelines by 'participating 
staff will be needed. ^ ' 

It should be recognized that absolute consistency of assessment can 
never b^ attained. Each school district will need to consldcfr the level of . 
inconsistency that is "tolerable." Clearly the consequences of incorrect 
judgments .will need to be considered and a 1>alance sflnck among such possible 
competlngi^^'ctors as student needs and societal ndeds. 

.Accg»tab ility/Transfer ability ^ ^ ' 

Although arrangements for establishing diploma standards will vary from 
school district to school district,^ other Institutions, agencies, or the like 



will evaiuajte th^'adequa^fy 9£'*t^9 evaluation and standard-setting prd^ 
cedures that l«id to gartlpular students earning diplomas. Sucl\ "outsiders" 
are likely to* raise quettlohd' about the evaltiation process and level of — 
standards employed. , By doing a good job of documenting aad explaining ^he 



.procedures that were followed in 4esigt^lhg evjbluation methods and in . 

\ • ' ^ - ■ '. 

establishing ancl applying appirpprlate^standards, slti instc^ltution can 

' ' ' - •' * 

Increase the acceptability or predib^Llity <of the diplomas that are grante^ 

% The problein of demonstrating that high enough standards were indeed 
followed Is a mote difficult one. Critics are fond of pointing oat examples 
of high school graduates wh^ca&pot read,* college graduettes who cannot / 
write, or provisional school graduates who dannot function effectively in 
the profession for which they were suppo^d^ trained.^ The best an indivi- 
dual Institution can i)robably do is to define clearly th^ planning, Ixaplemen- 

tation, and monitoring procedure >that it uses. The. greater the degree 

y ' ^ ' ' ' / ^ 

adherence to its policies that can be attained, and th\is the greater, the 

consistency in evaluation and staX^^ds application, that is , achieved, the less 

the Justification for criticism. 

' ,^ ^ ' ^ , ' • ^ - 

Student Satisfaction ^ , ^ 

In addition to meeting the criteria cfted above, 'an adequate 

. #- 

evaluation and standard-setting anyngement should satisfy thc^!^ majority. \ ^ 
x)f the students who have been evaluated. Provision needs ta be made for ✓ 
periodic f allow^up -surveys of students who have gonfe through the competeiffiy 
assessment ^ process, ^o v^at'extent i^o the students feel that they were 
prepared adequately for the assessment procedures that wete applied? Did 
the standards^ seem to b^ fair and realistic? Did they find (hat preparing 
for ancf experiencing the assessment w^s a valuable learning experience?- 



Was. there an air of ^erlousftess. about the enterprise that was 'cons Is tent ' 
with Its degree of Importance to the student's ftiture .educational program? 
Although students who do not ineAt standards Inltl^tlly can be expecte4 to 
be disappointed, they should be able to recognize the reasonableness of 
decisions made^ If they* do ^ot\ further lnVestlgatlQ^^|ft|g|| adequaqy of 
the sys.t«m .and its application, would clear ly^be called roi^.- 



METHODS OF SETTING COMPETENCY STANDARDS .V 
/ The growth of Interest In setting compet^cy standards. for the hlgb - 
school' diploma has caused many measurement professionals to examine past' 
work in the drea of standards setting. An article by John Meskauskas «ln the 

Winter (1976) Issue of Review of Educational Research will be very hilpful " 

^ ' *> 

■* - * ' ' ' , 

l#s this "Connection. Meskauskaa X1976) reviewed a'nuxti{>er of standards-set ti^ 

mo.del^ls- tiiat* have been applied In ctlte'rlon-referenced testing situations, 

where cont^nt areas have been tightly specified; 'He pointed out that dlf- 

ferent conceptions of the nature of "mastety" lead to different evaluation 

►* ^ 

y^rocedures, but that esctenslve practical applications are lacking for almost 

• - •■/" ^ >. 

; . • **~''* . • . - ^ 

' all*|methods. Another useful resource Is a paper by Loretta Shepard, pr^ 



sented at last years NfME meeting (Shepard* 1976) , which make^ some practical 
.recommendations on standard- set ting approaches. 



, The- discussion of methods of setting competency st;andards that follows 
is based odv the assimiptioh that three- factors will contribute ^c^l^l^^ly to 

\ ^ * , 

any school district's^ actions: . . ' 

• The ejcte^^l requirement's^ of other InsCl^tutions; agencies and . 

the likev e*^., state departments of education ' ^ 

/ ' • The basic educational mission and objectives of the school 
district . . • 

*'The personal develroiment goals of Individual students 



The weight g±ijr^ tetany oAe of these, factors will vary from 
' sibi^ition to 84tuati*on, but alj. will influence decisions about policy 
and practice In the- area of competeQcy standards for the diploouM, There 
will always be interactions among the factors, an(d there will be times 

h when serious conflicts will need to be resolved, as when -a student^s 

' \ ^ • • ' ^ ' 

personal devel6pmen^t goals do not medh with state requirementa or with, any 

of the alternatives offered by a dxrhool district. 

One basic determination that each. institution wlU need to make 
concerns the number pf domaiqs wljthl^ which it is prepare^ to set diploma 
standards and assessments: The larger the number of separate areas for 
which standards are 3et, the greater the proportion of students who 'will 
baye difficulty meeting these standards « cfearly, critical dedisiohs will , 
need to be made regarding the respective roles of successful course com- 
pletion aiKl Independent competency assessment in satisfying diploma re- 
Ijuirements. * ^ ' . 

Onbe the areas to be assessed have been identified and possible . 
assessment procedures selected, an approach to setting competency standards 
must be*chosen. ^This section describes and evaluates* seven possible methods:* 

• Development of Competence Scales * ^ 

• Traditonai or Arbitrary Method . . , 

• Review of Test Questions of Tasks \ 

• Identification of Prerequisite Skills ^» 
^ Use of Normative Data^ 

*; Relationship to Course Performance 

• Use of Preselected Groups 

' ' . • ^ ^ ■ ■ ' 

Development of Conlpetence Scales , 

There has been a good deal of work on ^he setting of performance 



Standards relatlngy^ training t>rograiii^ or licensing /certification settings 



In which evaluators are checking for the presence or absence of well* 

defined skills. 'The evaluation of the broader educational objectives of 

high schools will freqifently require more than a summlng-olSrunits approach. . 

|n some instances it may pvov^ more productive to develop competence scales 

jthat include: - ^ 

^\ , * Definitions of a series .of functional competi^nce levels < 
• (usually 3 to'5) ' • . *^ ' - 

-(Djlscriptions of the kind of performance that should Se 
associated with etch level 

* Illustrations, (examples, models) of each level . 
The degree of elaboration of these functional competence levels would 
necessarily vary both as a function 'Of the amount of student* and faculty 
time that could be devoted to their development and as a function of the 
amount of available Information on the qualities of student learning that 
should be dealt wlt^ in the scale. The initial. step in developing a com- 



petence scale might.be to list all the areas of knowledge • and skill (mat 

» 

shotild be considered. A next step could be to determine what level of 

^' - / 

knowledge or proficiexicy a person well above^ minimal competenc'^f might have. 
Such a step would provide the start fbr the high end of this competency^- 



scale. The other end might be develo 



ped by consideriiatg th^ absolute minimum 



skills that would be considered adequat^.* Through a combination of armchair 
speculation^ expert review, ancf trial use, a competence scale could be 
developed that evaluators would use as a guide to judgments about students. 

It would be a ^heroic enterprise to create^ a full spectrum of competence 
scales for every possible component of the li^igh school experience. * Instltu- 
tions comld move towards such a goal, though,, by developing competence scales 
very high priority objectives. It might be possible to draw on existing 
records of student performance and on the commex^s and experience of the. 
Staff and any outside evaluators. Pse of af^oon^tence-scale approach over 



t 



^ over a period of* years could result in the' development of ^ file of models 



of scales with' notes on €heir use. that would, help -evaluators facing future 
scale-def iniiion problems • 

A partial sample of the type of cdmpetenc^scale that has been, 
described can be fotgj^in the language proficiency scales developed tiy Che 
IT*. St^ Foreign Service Institute' and used for many years with Peace Corps 

volunteers: ' . ' ^ 

• * > 

Elementary Proficiency y 

USpeaking-t. Able to*;at1sfy routine travel n^s and minqmun courtesy 
uirements. ^Can ask -And' answer questions on topics ^very, famlTiar to him;- . ' 
within the scope of hi^«Vtof limited language experience can urrderstand sim- 
ple ^luestions and staftro^nxs, allowing for slowed speech, rep€t1t1or\ or para- 
phrase; speaking vocabuTffry inadeqtate^ to express anything but the most 
elementary needs ;, errors In immunciadbn and* granmar are frequent, but can 
, be understood by. a nathve speaker used to dealing with foreigners attempting 
to'speak his language; while topics which are "very fami 1 lar"' and elementary' 
needs var^ considerably /row individual to individual, any person at the 
. S-1 level should be-able to order a stmple nfieal ; ask for stielter or lodging, 

ask an^ give simple directions, make purchases, and teTl time,. 

• • * . / * 

Readfng-1. Able to read some person^^ and place names, street signs, 

offic,e and shop designations, numbers, and isolated words and phrases. Can 

.... . 



jjecognize^ all thfe ljette« ih the printed version of art alphabetic system 
and high-frequency ftlements |)f a syMabary or a character system.. 

Limned Working Proficiency - ' * . - 

Speak1ng-2. Able to satisfy routine social demandi and limited work 
.requirements. Can handle with confidence but not'wit^i facility most social 
situations includinj^introductions' and casual convensati^ons about current 
events, as well a? Work, ^family , and- autobiographical Information; can* han- 
dle limited 'V^ork requirements, needing help in ha/WlIng any complications . 
or difficulties; t;an get the gist of most conv^fsatloos on non-technical • 
1 subjects (I.e., topics whicn require bo specialized knowledge). and fias a 
I . '; spedkin9' vocabulary sufficient to express himself simply with some circujn-. * 
I locutions; ^accent, though, often quite faulty. Is intelligible; can usuaUy. 

\ , han<^le jelenienta'ry constructions quite accurately but does not *have thoroigti " 
' or confident control of the granror. ^ 

Readlng-2.. Able to read s.imple prose, in a ^orm equfvalent to type-, • ' ^ 
. , ' script or pr1ntirH[, 'on subjects withtn a familiar context. - With extensive 

^ , use of a dictionary can get tlje gerferal sense of rogtlfte .business letters, ' 

international ne|W5 Items., or articles In technical fields withjn his com- * 
petence. • ' , < 

• Minimum Professional Proficiency . 

/ , Speakl'ng-3. Able to speak the language with sufficient structural ac- 

curacy and vocabulary to participate- effectively in mest forma.1 and Informal \. 
conversations on practical, social, and professional topics. Can discuss 



• / 
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particular: Interests and spefclal flil^s of cbrfpetence with reasonable ease; 
comprehension is ^ulie compTete for a normal rate of speech; vocabulary is 
broad enough that he rarely has to jrppe for a word; accent may be obvious- 
ly foreign; control ,of grairmar good; errors never interfere with understand- 
ing and rarely disturb the native speaker. 

Reading-3/ Able to read s^and^nd newspaper itoms^ddressed to the gen- 
eral reader, routine corre$t)ondence, reports and tec^inical iwterial in his 
special .field. Can grasp the essentials of articles of the above types 
thout ysing a d1cti(^ary; for accurate understanding moderately frequent 
of a dictionary isfrequlred. Has occasional difficulty with unusually 
complex structures! and low-frequency idioms. 

fessional Proficiency 



Speaki^fl-4. Able to use the language fluently and accurately on all 
levels norma riv pertlnertt to professional needs. Can understand and parti- 
cipate irr any conversation within^ the range of his experience with a high 
degree of fluencj^nd, precision of vocabulary; would rarely be taken for a 
native speaker^ bJ^can respond appropriately even in unfamiliar situati6nsu 
errors of pronunciation and granmar quite rare; can handle informal int6r- 
pretinjg^ from and IntoN^e language. 

Reading-4. AtJle tKread all styles and forms of. the .langua'ge pertinent 
to professional nee^, W\th occasional use of a dictionary can read mod- 
erately difficult -jjrbsi re*^*ly In any area directed to the general reader, 
and all materia] 1n hls'specS^l f^eld including official and professional 
documents and correspondence ;\an read reasonably-legible handwriting with- 
out difficulty. ^ ' J * ' 

Native or Bilinqualv^ProfiCiency 

Speaking-5. Sj^aklnd proficieXcy equivalent to that of, an educated 
native speaker,, Has* coniplete fluency^in the language' such that his speech 
^ on all levels is fully acc$fpted by eduXated native speakers in all of its 
features, Including breadth of vocabulary and Idiom, col Ipquial Isms, and 
pertinent cultural references. , 

Reading-5. Reading" proficiency equivalent to that of .an educated na- 
tive. Can read extremely difficult and abstract prose', ^s well as highly 
colloquial writings and the classic literary forms of the language. With'^ 
' varying degrees q^dlffl^lty can r^ad all normal kinds of handwritten 
.documents. 



Traditional or Arbitrary Method 

^Uhen dealing with an examinatloaoi or set of ^etformvce tasks, the 
worst oft<all possible .methods o^ setting standards Is to choose an arbitrary 



est of i 



periBTOtage of correct responses 



because that percentage has tr>adltlonally 



been associated with a "passing' 



score. The method has Its roots In tha 



percent grades which were used. a{ 
until about 1920. Unfortunately 



llmost unl 



universally In schools and coll&gbs 



this ap{)roach (the "judgment uncontaminated 



by data" method) Is the most frequently used process* for setting .standards^ 



The* answer to the Question '*What s)eems right^ for a passing score?" usually 
represents a comprciQilse between tho^se who desire as- high a percentage as 
possible to show theli^ interest in quality education and those who are 
aftald that their standards wl^l be impossible for the bulk of students to 



meet. " 

^ ^ Clearly, nd method that falls to ta^e Into account tt^^e^ature df. the 
iLamlng experience y te^t questions or^ tasks, and the stiidents' backgrounds 
l8 satisfactory. A standard such ks 65 percent or 80 percent correct miglit 
be reasonable for certain tests for certain groups of students; It may be 
w|.ldly Inappropriate for other tests or sets' of performange tasks with other, 
students. As a rule of thumb > blanket performance stand.ards should be ' 
avoided . . ' 

\ • " , ' 

Review of Test Questions or Tasks ^ ' 

" . This method «ets a standard for each set of questions or performance 

tasks used for a specific competency area.^ Informed judges examine each 

question or behavioral statement of a critical jtask and decide if a mini- 

mally competent atadctlt could answer the question correctly or perform the 

task adequately. The standard is simply thew average ntimber of questloi^s 

or tasks that each Judge has determined should ^e answered correctly or 

performed at a specif led -li^vel of competence by minimally' competent students. 

One way to Improve this procedure In a situation Involving a test is 

to ask each Judge to state the probability that a minimally (Competent person 

• * • 
would answer each question correctly. The Judges review the records of a 

number of minimally competent persons and estimate the proportions of such 

students who gave the correct answer* The sum of these proportions represents 

the minimally acceptable^ score. Wftere task review is involved, judges will 



will need 'to decide on the weight to be given*, to different aspects sucK 
as the kind and degree of learning reflected In the produce or performance. 

Identification of Prerequisite Skills \ '* ^ 

.This methpd Is cl^rly of most use when the student's attalnmefit of 

: • •■ ^ ^ ■ • • 

an ultimate goal is unlikely^ unless certain prior "ehaBbltng" skills have 
been acquired. Success at advanced calculus, for example. Is not likely 
unless the ^tudent has mastered^ elementary algebra. Similarly^ success 

at discovering original proofs of mathematical theorems! in abstract .course 

' 1 ' - ^ 

work- is v^ry unlikely* unless the^stu^ent has previously \(1) learned how to 
reproduce (in his oxm words) standard textboojc proofs anfl (2) become 

thoroughly acqiial^t^d 'with several relatively cloncrete e^oamples of the 

\ •* \ . 

theoty so as to have\a stVong Intuitive feel for the absttaetlons. Although 

m^st courses of stiuiy do not enjpy the hierarchical precision of mathematics, 

the identififcation. of , the prerequisite skills ^nj^thpd is most efficient at 

points whfere decisiotis are made concerning a student's ability to undertake > 

a' new unit in a hierarchical sequence. of Ainlts.' 

This method usm' pooled judgments, but the ^decisions are made on the 

basis of whether «r not success olf a* question- or i;ask requires knowledge, or * 

sicllls that are essential to^ the future application^ of skills le^a^rned in ' 

the course ois program. The methcid, in practice, is quite similar to* thatlf>^ 

used in reviewing questions or tasks; it retain? the same advant^g^s and, 

problems. ^ But the "focus 1^ not on mskliig if a. minimally competent person 

WG^ld get^ a particular ques.tlon correct but rather if success on .a question 

or task is required^ before 8om:e specified future^ activity is possible; A 

taJjk analysis 'of the future activity can help to determine what behaviors 

are required in<the performance of complitated tasks. 



An exa^le of the kind of analysis that; can ^be particularly usfeful 
*in ^setting competency .requirements for the diploma was a survey of ^ national 

probabili^]j sample (N-5,096) of ailults aged 16 and over.. These adults were 

J, • 

asSkd to describe the kinds of reading performed in ' the ^^ourse of a nprmal 
day, theS ^amount , of time spent in carryingfout the reading described, and the 
iiiq)ortance attached to the various kinds of reading ^performed. Results of 
this survey were published in Reading Activities of American Adults (Sharon, 
1972). ' ' ^ , t ' 

Use of Norma ti^^e Data ^ - ) ^ ^ 

A norm--ref erenced test or a siimmary 6f/v^he products and performances ^of • 
a group of 'students can be used to make decisions about minimal coifpetency ' 
levels. Even though this method has ^pirical elements, pooled judgment is . 
still needed at som/^ stage in the selection df a norming or comparison group. 
A group must be chosen which is sufficiently similar to the group of interes^t. 
In contrast to previously discussed methods in Which judgments concchmed 
queistions or tasks, judgments in this method concern whether or not a student 
is just minimally competent. The assessment proce<^ure is then applied to a 
group of such students. The mean score, or average performance of that group^ 
may be taken as the miniitium performance standard. Using normative data tends 
to insure the setting of a performance standard that is reallstip in terms of 
the percentage of the population *that is judged to have reached an adequate 
level of performance on the basis of the test.. This mettiod has 'the most ready 

V 

applicaticTi^^to situations in. which an objectively scored examination or per- 
formance on a standard set of task^ will be the basis, for decisions regarding 



a student's level of* competence. 



V' 



Relationship to Course Terformancgi 

^Thls method can b^. applied to decisions. abW't minimal competence whe 

) . the skills require^ can be related to the^content of a particular course. j 

I A test or other measurement procedure that has been selected or developed I 

to be. the basis fo^ con^etencre decisions Is admlnksteted W students pom- 

^ % pLetlng the course. K table or graph Is prepared IsHowlng the rfelationahlp, 

between test scores or.'perf ormance ratings and grades \eamed In tKe regular 

course* .^A table can take the form of a grid containing, a n^bep^^of celJLs. 

' Inside each cell Is 'a number based on the number o'f peopue who . had scares 

on the test anci .grades In the course corresponding to the\ position of the 

cell. U3ually the data are organized to allow direct readings of probablkLtles 

of earning a particular course grade given a test score (e.g., of those who ' ^ 

> . c \ ' • - ^ 

^ scored In the range 45-50 on the test, what percenta^ eamei^ a grade of B 

or better?) Alten^tlvely, scores on the examination could b4 related to 

course gradtes by means of a regression equation, which will glye a predicted 

grade for any given test. score. <Warren Willingly reviews possible methods ) 

^ ' of ^usdn^. course performance as a basis fot setting a performance standard In 
a ohaptar entitled "Exemption frbnr Re^uj-rem^nts Already Mastered" in the book 

^ College Placement and Exemption .) , * ' ' 

'"When the relatlonshlp-to^coumi^-performance metfhod is used, the choice- 
o^^^e t^st performance standard is made on the basis of the probability of . 
obtaining various grades. Judgi^t is, of course, necessary in sc^^ctlng an 
appropriate probablllt;y. One might, for example, set a performance standard 
80 Chat there Is a probability of .80 Chat the grade earned in a regular course 
'would have been C (or some other level) or better. 
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- Use of Preselec-t ed Groups ^ ' • ^ \ 

V ' This .method requires that; information independent of the particular 
assessment procedure being applied be used to Select one group of students 
who are above a, school's miniiium Requirement and anothet that are not. 
Instructors' judgments should be used to divide students inta the two groups*. 

In making these, judgments, it is essential, to differentia^te- competencies from 

• ' ii 

other factors that contribute to grades; e.g.,^* punctuality, consistent atten- 

, t'^ ' - • ^ 

dance 9 ' classroom behavior ^ etc.^ 

^ . '' ^ . 

The assessin^n^ procedure is then applied to. both groups^ and the^ 

distribution of scores obtained. A' compet;ency standard is then selected that 

> ' ' • . . 

best discriminates between the two groups. \ ^ ^ 

This method has the important advantage of inherent concurrent validation: 
The test is validated against Independent informed judgments of* student com* 
petency. Furthermore, this procedure, allows the use of classical' measures of 
test quality such as reliability, validity, item diacrlmlnation, and the like. 

The methods of setting cosqpetency standards that are described in this 
paper combine humanij>Q^e^t and empi|l:ical data^.in a systematic process. 
Regardless of the quality of the procedure, however, some error of measure- 
ment will result. This "error" is not attri^^^^le to arithmetical mistake^* ^ 
that could.be avoided by more careful* cialculations. Such error is related 
to the complexity, and variability of people and the difficulty of measurteg 
their skills and knowledge. If you were somehow able to achieve complete 
knowledge of each student 'cofflpeta[icies and had a^ perfect standard*settlng 
procedure, you would b^ abl^ to detect some d^i^ee of error in at^y operational 
system. 

There are two major types^^ errors possible In setting standards. If 



Error of Measurement! 
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ttie staxidards are set too high, the" probability increases that those who 
shpuld be donsidered competent \d.ll be classified as not competent. If the i 
standards are set top low, the jwrobability Increases 'that those who should ^ 
faiii will be passed. Clearly, the setting of standards must tak6 into 
*^:oimt*^lie costs or losses associated with misclassification of each type. 
In .every situation, basic ptriadpSles of fairness, realism, and consistency 
wl^X ^PPly^ but it wi^ll also b^^ necessary to consider the social implications 
of decisions about students^ competencies. An analogy might be made 'to testing 
applicants for drivers' licenses. Th^re an incorrect judgment that an aji'pli- 
cant havacquired certain important competencies ^ght haVe the effect of , 
enabling the individual to avoid some needed additional training, allowing an 
Incoinpetjent driver to emerge as, a hazard to others. 



A Ftiaal Thought 

The Job of setting appropriate performance standards is as' critical as 
it is difficult. Developers of evaluation procedures as part ^of diploma re~ 
quirements should allow sufficient time in their projects for a thoughtful , 
ana],ysls of the possible approaches. If^at all possible, more than one . 

method of setting standards sBould then be tried, fnd the results compared 

k ** • - 

and evaluated. After this evaluation; a decision can be m^de regarding the 

S ' 

method of methods that will be employed for each assessment procedure. 
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